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ABSTRACT 

Correlation coefficients are frequently reported in 
educational and psychological research. The robustness properties and 
optimality among practical approximations when phi does not equal 0 
with moderate sample sizes are not well documented. Three major 
approximations and their variations are examined: (1) a normal 
approximation of Fisher's Z, N(sub 1) (R. A. Fisher, 1915); (2) a 
student's t based approximation, t(sub 1) (H. c. Kraemer, 1973; 'A. 
Samiuddin, 1970), which replaces for each sample size the population 
phi with phi*, the median of the distribution of r (the product 
moment correlation); (3) a normal approximation, N(sub6) (H. C. 
Kraemer, 1980) that incorporates the Jcurtosis of the X distribution; 
and (4) five variations— t (sub2) , t(sub 1)', N(sub 3), N(sub4), and 
N(sub4)' — on the aforementioned approximations. N(sub l) was fcund to 
be most appropriate, although N(sub 6) always produced the shortest 
confidence intervals for a non-null hypothesis. All eight 
approximations resulted in positively biased rejection rates for 
large absolute values of phi; however, for some conditions with low 
values of phi with heteroscedasticity and non-zero kurtosis, they 
resulted in the negatively biased empirical rejection rates. Four 
tables contain information about the approximations, (Author/SLD) 
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Robust approximations to the non-nuU distribution 
of the product moment correlation coefflciei:t I: The Phi coefficient 

Abstract 

Correlation coefficients are frequently reported in educational and psychological research. 
The robustness properties and optimality among practical approximations when p ^ 0 with 
moderate sample sizes are not well documented. Three major approximations and their 
variations are <jxamined: a normal approximation of Fisher's Z» Ni (Fisher, 1915); a Smdent's 
t based approdmation. ti (Kracmer, 1973; Saraiuddin. 1970), which replaces for each sample 
size the population p with p*, the median of the distribution of r, a normal approximation, Ng 
(Kraeraer, 1980), which incorporates the kurtosis of the X distribution; and five variations (t2, 
ti', N3, N4, N4') on the aforementioned approximations. Ni was found to be most appropriate, 
though N6 always produced the shonest confidence intervals for a non-nuU hypothesis. All 
eight approximations resulted in positively biased rejection rates for large absolute values of p 
but for some conditions witii low values of p with heteroscedasticity and nonzero kurtosis 
resulted in die negatively biased empirical rejection rates. 

Key words: Pearson Correlation, Phi coefficient, Robustness, Non-null distribution 
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Introduction 

The distribution of the product moment correlation has been widely investigated. In 
particular, the mathematical and empirical properties of the null distribution (p = 0) and its 
robustness have been extensively investigated (Duncan & Layard, 1973; Edgell & Noon, 
1987; Fisher, 1915; Gayen, 1951; Kowalski, 1972; Kraemcr, 1980; Pearson, 1931, 1932; 
Ramussen, 1987; Zimmerman, 1986). 

Let r be tiie product moment correlation coefficient between two random variables X and Y 
with V degrees of freedom. Then, the test statistic for Hq: p = 0 is 

T(rlO,v)=vi/2r(l-r2)-i/2. [1] 
When the parent distribution is bivariate normal, [1] distributes exactly as Student's t with v 
= n - 2 degrees of freedom, and as N (0,1) for large sample sizes. 

The general consensus for p = 0 is Uiat die normal approximation is robust even with small 
sample sizes when the parent distribution is bivariate nonnormal (Gayen, 1951; Havlicek & 
Peterson, 1977; Norris & Hjelm, 1961; Pearson, 1931, 1932; Zimmerman, 1986) provided that 
p = 0 implies independence between the two variables (Duncan & Layard, 1973; Edgell & 
Noon, 1987; Kowalski, 1972). The exception is given to the contaminated nontial variables 
where p = 0 does not imply independence between the two variables. As an alternative to 
applying tiie normal-r theory to the correlations obtained from bivariate nonnormal 
distributions, Duncan and Layard (1973) have proposed a bootstrapping method for testing p = 
0. Ramussen (1987), however, has shown that the normal approximation based on Fisher's Z 
transformation using in testing non-null correlations is superior to the bootstrapping in testing 
p = 0 for a nonnormal distribution satisfying the indep^ idence condition. 

Compared to the extensive research on the null distribution of r, the studies on the non-null 
distribution are comparatively few (Fowler, 1987; Gayen, 1951; Haldane, 1949; Kraemer, 1973, 
1980; Samiuddin, 1970). Gayen (1951) has derived the mathematical expression for the 
approximate sampling distribution of r for nonnormal distributions by incorporating the 
information up to the 4th joint moments between X and Y. Although Gayen has shown that a 
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simpler approximation to his derivation is satisfactory, his approximation is of little practical 
value because it is still computationally cumbersome and it requires the specification of the 
population joint moments, which are rarely known to a researcher. 

The most frequently adopted strategy for p ^ 0 is Fisher's Z transformation, which is 
defined as 

Zr=l/21n[(l+r)/(l-r)], [2] 
and is approximated by Ni when sample sizes arc large. Ni is a normal distribution with 
H = 1/2 In [(l+p)/(l - p)]+ p/{2(n-l)} {1 + (5 + p2) /4(n.l)}. and [3] 
^ ^ {l/(n-l)){ 1 + (4 - p2)/2(n - 1) + (22 - 6p2 - 3p4)/[6(n-l)2]}. [4] 
A simpler but less accurate approximation, N3, is again a normal distribution with 
ji = 1/2 In [(l+p)/(l - p)], and [5] 
a2 = l/(n-3). [6] 
Gayen has reported that considerably influenced by nonnormality when p ?t 0 and the 
normal approximation shows a satisfactory fit only for very large sample sizes. As an 
alternative to Fisher's Z, Samuiddin (1970) and Kraemer (1973) have proposed another 
transformation, T. 

T(rlp, V) = (r - p) [(l - r2) (l - p2)]-l/2, [7] 
and it is approximately distributed as t with v = n - 2 degrees of freedom. The approximation 
ti replaces p in [7] with p*, tiie median of the distribution of r, ( David, 1938; Kraemer, 197;), 
T(rlp*.v)= vl/2(r-p*)[(l.r2)(I -p*2)]-i/2. [8] 

Kraemer (1973) has reported that ti is most accurate for I pi <> .7, while Ni is most 
accurate when p is near unity (Ipl S.80) by showing the maximum deviations of the 
approximate cumulative distribution from the actual cumulative distribution. When p* is 
replaced with r, t2 approximation is obtained. The t2 approximation is reportedly satisfactory 
for n > 25 (Kraemer, 1973). 

Yet another derivation, which is a large sample approximation of tl, is N4. N4 is 
approximately normal with the following mean and variance: 
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H = l/21n -p*)],ana 

tP- « {l/(n-l))+[ 2/{(n - l)(n + 1))] + [23 /r3(n-l)(n + l)(n + 3)}]. 



[9] 



[10] 



Kraemer (1980) has examined yet another large sample approximation, Ng. Ne is a large 
sample approximation of [7] which incorporates the kurtosis, X O14/ ^% - 3) of the X- 
distribution. The T statistic distributes approximately normal as 



An optimal approximation for p ;6 0 for bivariatc ncnnormal distributions has not been 
determined because a direct comparison among ti, Ni, and N^, has not been conducted. TTic 
optimality conditions reported in Kraemer (1973) are focused on the maximum discrepancy for 
the entire area of the sampling distribution and not on the tails or on the confidence intervals 
for frequently chosen coverages. Consequently. *he approximation with the largest maximum 
error in her study may not necessarily be the worst approximation in terms of the test sizes 
and the typically used confidence intervals. 

Because Ni, ti, N4, t2, N3,and require the knowledge of the population parameter p, 
they can be used in testing an assigned value of p. However, the size of p is rarely known to 
the researcher, therefore, these are not practical in setting the confidence intervals for the 
unknown p unless the variance is independent of the parameter. For N4, and N3, a confidence 
interval can be derived for an unknown p because their variance estimates depend on the 
sample size alone. 

For ti and N4, it is also possible to derive a correction factor for p which does not depends 
on the size of p. The median of the p*\ p', which is the median of the medians obtained from the 
sampling distributions of r for a set of given p and n, was derived for the current study. The 
values of p* are dependent on both p and n, but the values of p' are independent of the 
population p and they can be determined for each sample size. A table of the correction 
factors, p' -p are presented in Table 1. The researcher, based on the sample size, for example, 
n = 12, can tell that the confidence limits forp should be adjusted by 0.0138 from the confidence 



T(rlp,v) -N(0,l+p2x/4). 
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limits set for p'. The procedures using p' arc ti' and N4', each corresponds to ti and N4, 
respectively. As stated before, the procedure t2 uses p in place of p* in the equation for ti. 
Table 1. Correction factor for non-7, ero p given the sample size 



correcrion factor 

0.0138 

0.0069 

0.0501 

0.0039 

0.0028 

0.0015 

0.0007 



Gayen (1951) reported that Fisher's Z transformed variable was considerably influenced 
by the non-normality if p^^O. The relative efficiency and robusmess of these eight 
approximations is not well known. The results reported in Kraemer (1973) are based on 
normal distributions where ti was found to be the optimal approximation as long as p was not 
too large; for p> .80, Fisher's Ni approximation showed a better fit to the empirical 
distribution of r than ti. Again, these results did not compare all eight procedures, nor were 
they concerned with typically selected test sizes or confidence interval coverages. 
Furthermore, the results are not well cumulated for the product moment correlation for discrete 
variables, namely, the Phi coefficient. 
The Phi coefficienf 

Given X and Y are discrete variables taking on 0 and 1. let p be rhe probability ofl in X 
variable and q be the probability of 1 in Y. The range of the product moment coefficient, which 
is called the Phi-coefficient will be limited by the values of p and q. 



n 



12 
22 
32 
42 
52 
102 
.202 
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X 



0 



1 



Y 1 



P 



0 



(1-P) 



(1-q) 



q 



Kraemer (1980) examined the robustness properties of the asymptotic normal distribution 
theory based on three assumptions: linearity, homoscedasticity, and zero kurtosis. The Phi 
coefficients were presented as an example nonnormal distribution with linearity trivially 
satisfied and for p = q = 0.5, the homoscedasticity condition is also met. The kurtosis of the X 
distribution is reflected in. the variance of Ne as (1 + p2 x/4), where \ = jxVcj^ - 3 in general, 
and is equal to [1 - 6p(l - p)]/4p(l - p) when X is discrete. She showed that for p ?t 0, even 
with both linearity and homoscedasticity being satisfied, the normal approximation, Ng, 
showed sizable deviations from the empincal distribution of r and that this trend became 
pronounced as p increased. Furthemicre, her Monte Carlo study indicated that Ng, which 
incorporates the kurtosis of the X-distribution, was extremely discrepant from the sampling 
distribution of the 9 -coefficient when p ^ q, where heteroscedasticity also existed. The 
performance of the other seven approximations nor the perfomiance of with respect to test 
sizes and confidence intervals are not yet known. 



The purpose of this study is to examine robustness of the three approximations, ti, Ni, Ng, 
and their five variations, ti', t2, N3, N4, and N4', to the non-null disuibution of the product 
moment correlation coefficient, the Phi coefficient, when the parent distribution is bivariate 
nonnormal. This research extends the work on the effect of heteroscedasticity and kurtosis by 
Kraemer (1980) for the Phi coefficient. It may be argued that a Chi-square distribution is often 
applied to a function of the Phi-coefficient in testing <P= 0 and not a normal approximation to 
the Phi-coefficient itself, therefore, a direct normal approximation may not be needed in testing 
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non-null hypotheses either. However, as we recall that a Chi-square is the squared normal 
distribution, they arc both rooted from the same asymptotic assumptions. 

An investigation in this area is most useful in applied research, because (1) a researcher 
rarely encounters a perfectly bivariate normal distribution; (2) a researcher cannot always 
secure a large enough sample size to depend on the large sample normal-r thcoiy; and (3) a 
researcher in a substantive area is likely to be dealing with p ;t 0 rather than p « 0 and accuiuic 
confidence intervals are indispensable in cumulating research effons in the field. 

Method 

A computer simulation method is used to investigate robustness of eight approximations 
when p?6 0. Random numbers are generated using the International Mathematical Subroutine 
Libraries (IMSL, 1989). A computer program for this study was benchmarkcd against the 
tabled data reponed by Kraemer (1980, pp.173-174) for the <P -coefficient. All experimental 
conditions had 1000 replications for each of the following conditions: two nominal a levels : 
0.05, 0.01; four sizes of p: .2, .4, .6, and .8; and six sample sizes: 12, 22, 32, 52, 102, and 202 . 
The distributions with p = q = 0.5, p = q 0.5 ( p = q = 0.25 and p = q = 0.75), andp9tq(p = 
0.25 q = 0.75 and p = 0.50 q = 0,75) were generated using the definitions for 2 x 2 cell 
probabiUties presented by Hamden (1949) and Kraemer (1980). The values of p and q limit 
the range of possible p, therefore, certain combinations were not examined because it was 
impossible to generate. 

Results 

Along with the confidence limits, the length of the confidence interval, empirical test size 
for testing p =po (po^) at a = 0.05. the sample r^ax and r^in value (Caroll, 1954; Guilford, 
1949) which indicate the range of the sampling distribution of r, are reported in the tables. 
Because the 1% results were very similar to the 5% counterparts in their trend, the selected 
results for the 5% are summarized in Tables 2 to 4. 

As the sample sizes reached over 30, all eight approximations were practically the same. 
For low r values with p = q = 0.5, all approximations were equally good except for N6» which 
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consistently showed a positive test size bias. In general, Ne showed the shortest confidence 
intervals followed by Ni; and Ne showed the largest test size of all eight approximations. For 
<P 9fe 0 with p = q = 0.5, basically all eight approximations except Ne behaved similarly for the 
conditions tested (Table 2). showed a decisive positive bias in its test size which did not 
diminish until n > 40 in testing 9 = .20. Once <P S .40, all eight approximations started to 
increase their test sizes. As reported in Kraemer (1980), Ne did not well behave when 9 
became large. From this study, it is also clear that any other approximations ud not behave 
well for (p S .80 (Table 3). When p^, tj and Ni both increased their test sizes but they were 
still favorable to N6. Ne seems to show the inflated Type I errors and the tendency becomes 
worse for certain conditions of (p = 0.50 and q = 0.75) with n < 30. However, for some 
other heterogeneity condition such as p = 0.25 and q = 0.75. all eight approximations showed 
negatively biased test sizes (Table 4). 

Discussion 

Although uncritical extrapolation of the preliminary simulation results should be avoided, 
all eight approximations were very close even for small sample sizes. Ne consistently showed 
the shortest confidence intervals for all conditions, however, it also showed a larger positive 
bias in its test size in comparison to the immediate competitor, Ni. The unexpected result 
was that N6 consistently showed its poor perfonnance. Even the simplest approximation, N3. 
seemed to be more robust and accurate than Ne. Another surprize was that even with p = q = 
0.5, thus, the homoscedastic condition is met, all eight approximations were not satisfactory 
for the large absolute value of 9. A practical guidance may be to adopt N3 in general but there 
is no optimal approximation so far for large absolute values of 9. Currently we are 
investigating several alternative approximations for the conditions with relatively high values 
of 9. 
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Table 2. Eight approximations for the non-null correlation coefficient with p = 0.2 and p = q = 0.5 

sample size i\l tl N6 t2 tl' N3 N4 N4' 

(fmin , ft Tmax) 



N = 12 

(-.719, .208, .751) 

95% Confidence limits (-.357, .667) (-.377, .664) (-.353, .665) (-.367, .674) (-.381, .660) (-.365, .672) (-.368,-659) (-.372, .655) 

[Length of Cn [1.024] [1.041] [1.018] [1.041] [1.041] [1.037] [1.027] [1.027] 

Rejection rates at 5%a 0.061 0.057 .084 .057 .057 .057 .061 .061 

N = 22 

(-.789, .198, .808) 

95% Confidence limits (-.222, .558) (-.230, .556) (-.217, .555) (-.225, .561) (-.231, .554) (-.223, .560) (-.227, .554) (-.229, .552) 

[Length of CI] [.780] [.786] [.773] [.785] [.785] [.783] [.781] [.781] 

Rejection rates at 5%a .046 .046 .070 .046 .046 .049 .046 .046 

N = 32 

( -.808, .202, .845) 

95% Confidence limits (-.144, .506) (-.150, .504) (-.141,. 504) (-.146, .507) (-.151, .502) (-.145,-507) (-.148, .503) (-.150, .501) 

[Length of CI] [.650] [-654] [.645] [.653] [.653] [.652] [.651] [.651] 

Rejection rates at 5%^ 0.068 .068 .078 .068 .0650 .068 .068 .065 



a 95% SE of simulation = .01 35, providing (.0365, .0635) as a CI for the nominal level of 5%. 
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Tabic 3. Kght approximations tor the non-null correlation coefficient with p = 0.8 and p = q = 0.5 

sample size Nl tl N6 t2 tl* N3 N4 N4' 

frmin , i» fmax) 

N= 12 

(-.676, .875, .800) 

95% Confidence limits (.501, .935) (.473. .924) (.531, .929) (.487, .938) (.473, .924) (.489, .937) (.480, .923) (.480, .923) 

[Length of CI] [.434] [.451] [.397] [.451] [.451] [.448] [.443] [.443] 

Rejection rates at 5%a 0.320 .320 .342 .318 .320 .318 .320 .320 

N = 22 

(-.752, .795, .900) 

95% Confidence limits (.584, .908) (.574, .903) (.606, .901) (.580, .909) (.573, .902) (.581, .909) (.575, .902) (.575, .902) 

(Length of CI] (.324] [.329] [.295] [.329] [.329] [.328] [.327] [.327] 

Rejection rates at 5%a 0.154 .154 .389 .145 .154 .145 .154 .154 

N = 32 

(-.775, .800, .915) 

95% Confidence limits (.637, .896) (.631, .893) (.654, .890) (.636, .897) (.630, .892) (.636, .897) (.631, .892) (.631, .892) 

(Length of CI] [.259] [.262] [.236] [.261] [.261] [.261] [.261] [.261] 

Rejection rates at 5%a 0.253 .253 .266 .228 .253 .232 .253 .253 



a 95% SE of simulation = .0135, providing (.0365, .0635) p*^ CI for the nominal level of 5%. 
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Table 4. Eight approximations for the non-null correlation coefficient with p = 0.2 and p = 0.25 q = 0.75 



sample size Nl ll N6 t2 tl' N3 N4 N4' 

(Xmin , f » Tmax) 

N= 12 

(-.698, .194. .334) 

95% Confidence limits (-.392, .668) (-.412. .665) (-.390, .667) (-.403. .675) (-.416. .661) (-.400, .673) (-.403, .660) (-.408, .656) 
lUngthofCn ri.060] [1.078] [1.058] [1.078] [1.078] [1.073] [1.063] [1.063] 

Rejection rates at 5%a 0.026 .020 .034 .020 .020 .020 .026 .026 

N = 22 

(-.741. .205. .331) 

95% Confidence limits (-.228, .564) (-.237, .562) (-.226, .562) (-.231, .567) (-.238, .560) (-.230, .566) (-.234, .560) (-.236, .558) 
(UngthofCH [.793] [.799] [.788] [.798] [.798] [.796] [.794] [.794] 

Rejection rates at 5%a 0.023 .023 .027 .015 .023 .015 .023 .023 

N«32 

(-.777, .196. .330) 

95% Confidence limits (-.156. .504) (-.154. .503) (-.157, .505) (-.162. .500) (-.156. .505) (-.160, .501) (-.161, .500) (-.162, .500) 
[Length of CI] [.660] [.663] [.656] [.663] [.663] [.661] [.661] [.661] 

Rejection rates at 5%a 0.018 .017 .021 .017 .018 .017 .018 .018 



3 95% SE of simulation = .0135, providing (.0365, .0635) as a CI for the nominal level of 5%. 
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